Disentangled Graph Recurrent Network for Document Ranking

Abstract BERT-based ranking models are emerging for its superior natural language understanding ability. All word relations and representations in the concatenation of query document modeled self-attention matrix as latent knowledge. However, some knowledge has none or negative effect on relevance prediction between document. We model observable unobservable confounding factors a causal graph perform do-query to predict label given an intervention over this graph. For observed factors, we block back door path by adaptive masking method through transformer layer refine disentangled refinement layer. unobserved resolve do-operation from front decomposing into related unrelated parts decomposition Pairwise loss is mainly used ad hoc task, triangle distance introduced both layers more discriminative representations, mutual information constraints put Experimental results public benchmark datasets TREC Robust04 WebTrack2009-12 show that DGRe outperforms state-of-the-art baselines than 2% especially short queries.